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From  Multi-Agent  Simulation  of  Human  Behavior  in  Naval  Air 
Defense,  Figure  2:  Example  view  of  user  interface  to  ADC 
Simulation. 
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■  Robert  L.  Steele,  rsteele@oavalengweers.com 


Inside  the  Fall  Issue  of  The  Naval  Engineers  Journal 


Professor  Verma  and  Mr.  Ghadmode  authored  an 
article  entitled  “An  Integrated  Lean 
Implementation  Model  for  Fleet  Repair  and 
Maintenance,”  which  was  presented  at  the  ASNE  Fleet 
Maintenance  Symposium  2003  in  Virginia  Beach, 
Virginia.  The  paper  provides  a  survey  of  existing  lean 
implementation  models  for  ship  repair  and  mainte¬ 
nance  and  presents  a  new  model  as  well.  “Lean"  is 
defined  as  a  systematic  approach  to  identifying  and 
eliminating  waste  (non  value-added  activities)  through 
continuous  improvement  by  flowing  the  product  at  the 
puli  of  the  customer  in  pursuit  of  perfection. 

Dr.  Hsing-Chia  Kuo  and  Dr.  Hui-Kuo  Chang  from  the 
National  Cheng  Kung  University  in  Tainan,  Taiwan 
have  contributed  a  paper  entitled  “Application  of 
Symbiotic  Evolution-based  Fuzzy-Neural-Networks  to 
Fault  Diagnosis  of  Marine  Propulsion  Systems.”  The 
proposed  system  combination  of  fuzzy  modeling, 
back-propagation  training,  and  symbiotic  evolution 
function  auto-generates  its  own  optimal  fuzzy  neural 
architecture,  a  significant  advantage  over  previous 
time-consuming  manual  parameter  determination.  The 
authors  state  that  the  presented  design  is  useful  as  a 
core  model  for  more  advanced  computer  assisted  diag¬ 
nostic  systems  and  for  direct  application  in  maritime 
propulsion  systems. 

Mr.  William  M.  Johnson  who  is  the  project  manager 
for  open  architecture  standards  and  planning  in  PEO 
Integrated  Warfare  Systems  offers  a  program  article  on 
the  submarine  A-RCI  (acoustic-rapid  commercial-off- 
thc-shelf  insertion)  process. 


Lt.  Sharif  H.  Calfee,  USN  and  Dr.  Neil  Rowe  submitted 
a  paper  titled  “  Multi-Agent  Simulation  of  Human 
Behavior  in  Naval  Air  Defense.”  It  is  based  on  the 
AEGIS  Cruiser  Air-Defense  Commander  Simulation 
Program,  which  models  the  operations  of  a  combat 
information  center  team  performing  air  defense  for  a 
U.S.  Navy  battle  group.  Designed  primarily  to  assist 
personnel  in  air-defense  training  and  doctrine  formula¬ 
tion,  it  describes  the  factors  that  influence  performance. 

Drs.  Almet  D.  Alkan  and  Kayhan  Gulez  have  authored 
a  paper  entitled  “A  Knowledge-Based  Computational 
Design  Tool  for  Determining  Preliminary  Stability 
Particulars  of  Naval  Ships.”  In  this  study  the  authors 
established  a  robust  neural  network  structure  using 
design  data  from  22  naval  ships  resulting  in  a  design 
tool  for  determining  ship  preliminary’  stability  particu¬ 
lars  and  load  capacity. 

Dr  Gokdeniz  Neser  and  Mssrs.  Sinan  Songuler  and 
Mehemet  Emin  Tacar  submitted  a  technical  article 
titled  “Contributions  to  the  Marine  Use  of  Non-Crimp 
Reinforcement  by  Means  of  Strength  Tests.”  They  sur¬ 
mise  that  non-crimp  reinforcements  offer  the  advan¬ 
tages  of  lighter  structures  at  a  lower  labor  cost. 

Drs.  Ravi  Penmetsa,  Ramana  Grandhi,  and  Vipperla 
Venkayya  state  that  their  paper  “  Modeling,  Analysis, 
and  Uncertainty  Quantification  of  a  Lightweight 
Torpedo  Design,”  presents  a  robust  torpedo  design 
strategy,  which  provides  the  designer  insight  to  the  safe¬ 
ty  of  the  final  configuration.  A  torpedo  is  modeled 
using  finite  elements  and  its  static  and  dynamic  charac¬ 
teristics  are  analyzed  subject  to  variations  in  the  design 
parameters.  ■ 
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Multi-Agent  Simulation  of  Human 
Behavior  in  Naval  Air  Defense 

■  Lt.  Sharif  H.  Calfee,  USN  and  Neil  C.  Rowe 

Abstract 

The  AEGIS  Cruiser  Air-Defense  Simulation  Program  models  the  operations  of  a  combat  information 
center  team  performing  air  defense  for  a  U.S.  Navy  battle  group.  It  uses  multi-agent  system  technology 
and  is  implemented  in  Java.  Conceived  primarily  to  assist  personnel  in  air-defense  training  and  doctrine 
formulation,  the  simulation  provides  insight  into  the  factors  (skills,  experience ,  fatigue,  aircraft  numbers, 
weather,  etc.)  that  influence  performance,  especially  under  intense  or  stressful  situations,  and  the  task 
bottlenecks.  It  simulates  air  contacts  (aircraft  and  missiles)  as  well  as  the  actions  and  mental  processes  of 
the  watchstanders.  All  simulated  events  are  logged,  to  permit  performance  analysis  and  reconstruction 
for  post-scenario  training.  Validation  of  the  simulation  was  done  with  the  help  of  expert  practitioners  of 
air  defense  at  the  AEGIS  Training  &  Readiness  Center  (ATRC),  San  Diego,  California,  USA. 


Introduction 

ir  warfare  is  the  most  rapid,  intense, 
and  devastating  type  of  warfare  that 
the  U.S.  Navy  trains  for  and  is  a 
major  focus  of  the  operations  of  ships 
(Mairorano,  Carr,  and  Bender  1996). 

Because  of  the  fast  pace,  uncertainty,  and 
dangers  of  air  warfare,  the  air-defense  team 
of  “watchstanders”  must  train  extensively  in 
the  many  skills  needed  for  these  operations. 
The  needed  skills  involve  search,  detection, 
and  classification  to  determine  and  maintain 
identification  of  all  aircraft  and  surface  ves¬ 
sels  within  the  operational  area  of  the  ship’s 
“battle  group”  (or  coordinated  group  of 
ships).  However,  continued  advances  in 
speed,  maneuverability,  and  accuracy  of  anti¬ 
ship  missiles  have  reached  the  point  where, 
despite  defensive  countermeasures,  human 
watchstanders  can  be  unable  to  communi¬ 
cate,  coordinate,  and  react  quickly  and  cor¬ 
rectly.  Two  incidents  in  the  1980s  highlighted 
the  need  for  improvements:  The  USS  Stark 
was  attacked  by  two  Exocet  anti-ship  mis¬ 
siles  and  was  nearly  sunk,  and  the  USS 
Vincennes  mistakenly  shot  down  a  civilian 
Iranian  airliner  during  a  surface  battle  with 


Iranian  naval  forces.  Air  defense  does  require 
human  judgment  because  of  widely  varying 
geographical,  environmental,  and  tactical 
conditions,  and  thus  would  be  dangerous  to 
fully  automate  despite  interesting  attempts 
(Noh  and  Gmytrasiewicz  1998). 

This  paper  reports  on  the  AEGIS  Cruiser  Air- 
Defense  Commander  (ADC)  Simulation,  built 
to  model  the  performance  of  U.S.  Navy 
watchstanders  engaged  in  air  defense  (Calfee 
2003).  In  a  naval  battle  group,  which  consists 
of  the  aircraft  carrier  and  six  to  eight  sup¬ 
porting  warships,  the  air-defense  commander 
is  responsible  for  the  coordination  of  the 
force’s  resources  (ships,  aircraft)  and  efforts 
to  conduct  surveillance,  detection,  identifica¬ 
tion,  intercept,  and  engagement  of  aircraft 
within  the  operational  area  with  the  primary 
objective  of  defending  the  aircraft  carrier  (or 
other  high-value  unit).  While  other  work  has 
modeled  the  threats  in  air  defense  (Barcio  et 
al.  1995,  Bloeman  and  Witberg  2000,  Choi 
and  Wijesekera  2000,  Delaney  2001),  the 
ADC  Simulation  appears  to  be  the  first  to 
carefully  model  the  watchstanders  as  well. 

On  Navy  ships,  air  defense  is  done  in  the 
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Combat  Information  Center  (CIC),  which 
contains  consoles  for  activating  weapon  sys¬ 
tems,  configuring  sensor  systems,  displaying 
contact  tracks,  and  communicating  with 
other  ships  and  aircraft.  Methodology  for 
modeling  such  teams  has  become  increasingly 
sophisticated  in  recent  years  (Weaver  et  al. 
1995).  Figure  1  gives  the  organization  of  the 
11 -person  CIC  air-defense  team  on  tire 
AEGIS  cruiser  serving  as  the  air-defense  com¬ 
mander  for  the  battle  group  forces. 

The  air-defense  watchstanders  are: 

■  The  force  tactical  action  officer  (FORCE 
TAO)  controls  air  defense  for  the  battle 
group  and  is  responsible  for  major  deci¬ 
sions  such  as  contact  classifications  and 
weapons  releases.  There  is  only  one 
FORCE  TAO,  who  is  located  on  a  battle 
group  ship.  The  FORCE  TAO  works 
directly  for  the  ADC  who  in  turn  reports 
to  the  battle  group  commander. 

■  The  force  anti-air  warfare  coordinator 
(FORCE  AAWC)  coordinates  the  move¬ 
ment  and  assignment  of  friendly  aircraft 
and  orders  the  weapons  employment  by 
ships  in  defense  of  the  battle  group.  The 
FORCE  AAWC  works  directly  for  the 
FORCE  TAO. 

■  The  ship  anti-air  warfare  coordinator 
(SHIP  AAWC)  directs  aircraft  detection 
and  classification  for  a  single  ship  in  sup¬ 
port  of  the  air-defense  effort  within  its  air¬ 


space  and  manages  the  identification 
process.  The  SHIP  AAWC  coordinates 
directly  with  the  FORCE  AAWC. 

■  The  ship  tactical  action  officer  (SHIP 
TAO)  leads  the  CIC  watch-team  air- 
defense  effort  for  a  single  ship  and  coordi¬ 
nates  directly  with  the  FORCE  TAO. 

■  The  missile  systems  supervisor  (MSS)  fires 
(under  orders)  the  ship’s  surface-to-air 
missiles  and  the  self-defense  close-in 
weapon  system. 

■  The  Red  Crown  watchstander  (RC)  moni¬ 
tors  friendly  aircraft  for  the  battle  group. 
The  RC  coordinates  directly  with  the 
FORCE  TAO  and  FORCE  AAWC. 

■  The  electronic  warfare  control  officer 
(EWCO  or  EWS)  is  responsible  for  the 
operation  of  the  electronic  emissions 
detection  equipment  used  to  detect  and 
classify  aircraft. 

■  The  identification  supervisor  (IDS)  does 
identification  friend  or  foe  (IFF)  chal¬ 
lenges  on  unknown  aircraft  and,  when 
directed,  initiates  query  or  warning  proce¬ 
dures  for  contacts. 

■  The  radar  systems  controller  (RSC)  oper¬ 
ates  the  SPY-1  A/B  radar  systems,  the  pri¬ 
mary  means  by  which  aircraft  are  detect¬ 
ed  and  tracked. 

■  The  tactical  information  coordinator 
(TIC)  operates  the  tactical  digital  infor¬ 
mation  links,  which  communicate  tactical 
data  among  the  ships  and  aircraft  in  the 
battle  group. 

■  The  combat  systems  coordinator  (CSC) 
monitors  the  status  of  the  combat  systems 
that  support  the  CIC  and  repairs  them  as 
necessary. 

Related  Work 

The  tactical  decision-making  under  stress 

study  explored  the  causes  of  the  USS 

Vincennes  incident  (Morrison  et  al.  1996). 

Some  problems  were  identified  with  the 
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short-term  memory  limitations,  such  as  for¬ 
getting  and  confusing  track  numbers,  forget¬ 
ting  and  confusing  kinematic  data,  and  con¬ 
fusing  tracks  of  contacts.  Other  problems 
were  related  to  decision  bias,  such  as  carry¬ 
ing  initial  threat  assessment  throughout  the 
scenario  regardless  of  new  information,  and 
assessing  a  contact  from  past  experiences. 
This  work  also  suggested  how  to  improve 
command  center  display  consoles.  For  real¬ 
ism,  the  ADC  Simulation  approximates 
some  of  these  cognitive  errors. 

Other  work  examined  the  cognitive  aspects 
of  the  threat-assessment  process  used  by 
naval  air-defense  officers  during  battle  group 
operations  (Liebhaber  and  Smith  2000).  This 
indicated  that  watchstanders  had  possible- 
track  templates,  derived  from  a  set  of  twen¬ 
ty-two  identifying  factors,  which  they  used 
to  classify  contacts  and  calculate  threat 
assessments.  Some  of  the  most  promising 
factors  were  electromagnetic  emissions, 
course,  speed,  altitude,  point  of  origin,  flight 
profile,  intelligence  information,  and  identifi¬ 
cation  friend  or  foe  (IFF)  mode.  This 
research  was  very  helpful  in  development  of 
the  ADC  Simulation. 

Situational  awareness  was  also  identified  as  a 
primary  concern  during  task  analysis  for  the 
Joint  Maritime  Command  Information 
System  (Eddy,  Kribs,  and  Cohen  1999).  It  was 
affected  by  (1)  capabilities,  (2)  training  and 
experience,  (3)  preconceptions  and  objectives, 
(4)  and  ongoing  task  workload.  “As  task 
workload  and  stress  increase,  decision-makers 
will  often  lose  a‘Big  Picture’  awareness  and 
focus  on  smaller  elements.”  This  was  also 
incorporated  into  the  ADC  Simulation. 

Computer  games  have  simulated  naval  com¬ 
bat.  Strike  Fleet™  in  1987  was  an  early  video 
game  that  simulated  naval  battle  group  oper¬ 
ations.  Fifth  Fleet™  was  introduced  in  1994 
and  immediately  set  a  standard  for  the  accu¬ 
rate  depiction  of  naval  operations  and  realis¬ 
tic  game  play.  But  the  Harpoon™  Series 


games  (Strategic  Simulations,  Inc.)  have  been 
the  most  popular  naval  games,  and  they 
have  spanned  nearly  fourteen  years  from 
Harpoon  1™  published  in  1 989  to  Harpoon 
3™  in  2003.  The  game  engines  were  based 
on  a  realistic  war-gaming  and  operational 
analysis  model  designed  by  the  creator,  Larry 
David,  a  former  naval  analyst  and  author. 
They  featured  very  accurate  representations 
of  platforms,  weather  phenomena,  weapon 
systems,  geography,  friendly  and  opponent 
tactics,  as  well  as  believable  scenarios  and 
campaigns  based  on  current  and  future  polit¬ 
ical  and/or  actual  conflicts.  Some  military 
and  military-affiliated  organizations  have 
used  the  game  as  part  of  their  training, 
including  the  United  States  Air  Force 
Command  and  Staff  College,  U.S.  Naval 
Institute,  Australian  Department  of  Defense, 
and  the  Brazilian  Naval  War  College. 
However,  these  games  have  not  been  used 
much  by  the  military  because  their  primary 
purpose  is  entertainment.  They  lack  key  fea¬ 
tures  needed  such  as  psychological  models  of 
personnel  and  comprehensive  logging  of 
events  to  help  formulate  lessons  learned. 

Several  systems  have  been  developed  to  aid 
air-defense  personnel  in  their  duties.  The 
Area  Air-Defense  Commander  Battle 
Management  System  was  developed  for 
more  effective  coordination  of  air-defense 
planning  and  execution  for  multi-service 
(i.e.,  Army,  Air  Force,  Navy,  and  Marines) 
and  international  operations  (Delaney  2001), 
It  develops  and  executes  a  theater-wide  air- 
defense  plan  providing  an  integrated  view  of 
the  battlespace.  Its  focus  is  threats,  not  the 
personnel  responding  to  them. 

The  Multi-Modal  Watch  Station  (MMWS) 
Program  developed  specialized  watchstation 
consoles  with  improved  human-computer 
interface  designs  to  help  watch  teams  during 
battle-group  air  defense  and  land-attack 
operations  (Osga  et  al.  2001 ).  The  MMWS 
consoles  were  designed  to  increase  usability 
and  learnability  and  decrease  the  potential 
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for  information  overload  and  errors. 

Research  conducted  extensive  interviews  and 
console  evaluations  with  air-defense  subject 
matter  experts.  The  V1MWS  consoles  cor¬ 
rected  interface  problems  in  the  current 
AEGIS  C1C  consoles  which  caused  errors, 
information  overload,  and  loss  of  situational 
awareness.  These  consoles  reduced  the  size 
of  the  air-defense  team  by  two  or  three  peo¬ 
ple  while  increasing  performance. 

The  Battle  Force  Tactical  Trainer  System  was 
designed  for  the  Fleet-wide  training  of  naval 
units  by  providing  each  ship  with  a  system 
using  the  existing  CIC  console  architecture 
(Federation  of  American  Scientists  2003). 
High-fidelity  scenarios  can  inject  actual  sig¬ 
nal  information  into  the  ship  combat  sys¬ 
tems  to  simulate  reality.  The  BFTT  System 
can  simulate  an  entire  fleet  of  ships  and  their 
staffs  and  support  war-gaming  exercises. 

Although  the  ADC  Simulation  builds  on  pre¬ 
vious  research,  it  has  several  unique  features: 

■  It  focuses  on  the  decision-making  and 
other  mental  processes  of  the  watch- 
standers. 

■  It  explores  the  influence  that  a  watch- 
stander’s  proficiency  has  on  the  perfor¬ 
mance  of  the  air-defense  team. 

■  It  allows  the  user  to  configure  a  wide 
range  of  attributes  to  determine  their 
effects  on  performance. 

■  It  allows  the  user  to  examine  the  assump¬ 
tions  of  the  simulated  watchstanders  and 
compare  them  to  the  truth. 

■  It  provides  views  of  the  simulation  on  a 
variety  of  time  scales. 

■  Its  multiagcnt  system  that  simulates  the 
watchstanders  provides  for  a  realistic 
reproduction  of  human  behaviors. 

■  It  logs  several  kinds  of  data. 

■  It  represents  a  key  piece  for  a  naval 
wargaming  simulation. 


Design  of  the  Simulation 

Design  of  the  ADC  Simulation  was  done  in 
six  steps.  First,  using  the  Human-Computer 
Interface  User-Centered  Design  (UCD) 
Process,  comprehensive  interviews  were  con¬ 
ducted  with  five  experienced  air-defense  sub¬ 
ject-matter  experts  from  the  AEGIS  Training 
and  Readiness  Center  Detachment  in  San 
Diego,  California,  and  the  Fleet  Technical 
Support  Center  Pacific,  to  collect  data  about 
battle-group  air  defense.  These  personnel 
possessed  five  to  fifteen  years  of  naval  air- 
defense  experience,  and  were  considered 
experts  by  their  peers  and  the  U.S.  Navy. 
Second,  the  materials  of  the  simulation  were 
determined  from  this  data:  agents,  objects, 
and  attributes.  Third,  the  relationships 
between  agents  and  between  agents  and 
objects  were  explicitly  defined.  Fourth,  the 
tasks  and  actions  for  each  agent  and  object 
were  defined.  Fifth,  the  simulation  was  built 
from  this  information,  including  the  neces¬ 
sary  control  actions  (see  Table  1).  Sixth,  the 
simulation  was  debugged  and  some  minor 
adjustments  were  done. 


The  ADC  Simulation  Program 

The  Simulation  Interface 

The  ADC  Simulation  is  implemented  in  Java. 
Figure  2  shows  an  example  view  of  the 
graphical  user  interface.  The  center  of  the 
screen  shows  the  locations  and  statuses  of 
the  aircraft  and  missile  contacts.  The  left 
side  shows  detailed  information  about  the 
contact  currently  in  focus;  the  right  side 
gives  information  about  the  watchstanders. 
The  top  of  the  screen  and  the  lower  left  pro¬ 
vide  controls  for  the  simulation. 

The  center  of  the  screen  shows  the  move¬ 
ment  of  contacts  across  the  airspace  and  uses 
colors  and  symbols  to  indicate  their  status, 
both  perceived  and  real.  The  Persian  Gulf  in 
2002  was  used  in  the  prototype.  Aircraft 
contacts  are  friendly  (U.S.  aircraft),  neutral 
(commercial  aircraft),  unknown ,  suspect 
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(potentially  hostile),  and  hostile  (known 
Iranian  and  Iraqi  aircraft).  A  contact  with  a 
green  background  means  it  has  not  been 
processed  by  the  radar  systems  controller, 
and  yellow  means  the  FORCE  TAO  has 
classified  the  aircraft  incorrectly.  This  display 
is  interactive  and  the  user  can  select  contacts 
to  view  their  data  and  modify  attributes. 

The  left  side  of  the  screen  shows  two  sets  of 
data  about  the  currently  selected  contact,  the 
actual  data  and  the  data  perceived  by  the 
simulated  CIC  team.  Buttons  provide  short¬ 
cuts  for  starting,  stopping,  pausing,  or  con¬ 
tinuing  a  scenario.  They  also  control  the 
time  compression,  the  ratio  of  simulation 
time  to  standard  time.  The  right  side  of  the 
screen  gives  a  picture  of  the  CIC  team. 
Watchstander  icons  (the  circles)  can  be 
selected  to  display  associated  attributes. 

Some  watch  standers  collect  and  assess  sensor 
information;  others  make  decisions;  and  oth¬ 
ers  carry  out  defensive  and  offensive  actions. 
Each  watchstander  has  a  mental  activity 
indicator  giving  the  status  in  regard  to  the 
current  task  as  either  high,  medium,  or  low 
(indicated  with  colors)  to  give  an  indication 
of  watchstander  stress. 

Contacts 

The  aircraft  or  missile  contact  is  the  funda¬ 
mental  object  in  the  simulation.  It  has  a 
track  number,  point  of  origin,  course,  speed, 
altitude,  radar  cross-section,  electronic  signal 
emissions  (based  on  radar  type),  and  IFF 
(identify  friend  or  foe)  mode.  It  also  has  ana¬ 
lytic  attributes:  whether  it  has  been  detected 
by  the  CIC,  whether  it  has  been  evaluated, 
the  last  time  it  was  evaluated,  whether  it  is 
approaching,  and  its  threat  status. 

Contacts  are  created  by  the  simulation  as  it 
runs.  Friendly  aircraft  are  generated  by 
orders  of  the  FORCE  TAO  or  FORCE 
AAWC  agents.  Other  aircraft  are  generated 
at  random  based  on  the  attributes  of  contact 
density  and  hostile  contact  level.  Surface-to- 


Table  i 


ADC  Simulation  user  actions 

TASK#  TASK  NAME 

1  Open  scenario  menu 

2  Open  watchstander  attributes  menu 

3  Open  CIC  equipment  setup  menu 

4  Open  scenario  doctrine  setup  menu 

5  Open  scenario  external  attributes  menu 

6  Open  simulation  logs  menu 

7  Change  the  maximum  time  it  takes  a  watchstander  to  complete  a  task 

8  Display  data  about  a  contact 

9  Display  data  about  the  FORCE  TAO  watchstander 

10  Open  a  contact's  pop-up  options  window 

u  Open  the  FORCE  TAO  pop-up  options  window 

12  Increase  the  time  compression 

13  Pause  the  simulation 

14  Set  the  situation-assessment  skill  level  to  Expert  for  the  FORCE  TAO 

15  Set  the  fatigue  level  to  Exhausted  for  the  RSC 

16  Set  the  SPY-iB  radar  equipment  readiness  level  to  Non-Operational 

17  Set  the  ADC  doctrine  query  range  to  30  nm  and 
warning  range  to  20  nm 

18  Set  the  scenario  threat  level  to  Red 

19  Open  the  scenario  event  log 

20  Open  the  SLO-32  system  status  log 

21  Set  the  watchstander  fatigue  levels  to  (0.5, 0.7, 0.9) 

22  Change  the  maximum  time  for  the  FORCE  TAO  watchstander  to 
complete  a  task 

23  Change  the  speed  of  the  hostile  air  contact  to  500  kts 

24  Change  the  FORCE  AAWC  experience  attribute  to  Expert 

25  Change  the  link  equipment  status  to  Partially  Degraded 


FIGURE  2 

Example  view  of  user  interface  to  ADC  Simulation. 
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air  missile  and  anti-ship  missile  contacts  are 
created  by  orders  from  either  the  CIC  or 
hostile  aircraft.  In  the  simulation,  contacts 
fly  piecewise  linear  paths  with  landing  and 
descent  behavior  at  the  ends.  They  may 
respond  to  queries  and  warnings  from  the 
CIC,  and  may  retreat  or  alter  their  courses 
accordingly.  They  may  also  experience  loss 
of  their  radar  or  IFF  systems  with  a  certain 
probability. 

The  neutral  aircraft  fly  directly  between  two 
points.  The  friendly  aircraft  are  fighters  or 
support  directly  controlled  by  the  CIC  team. 
They  depart  and  return  from  friendly  bases 
and  carriers,  and  conduct  visual  intercept, 
identification,  and  possible  engagement  of 
other  aircraft.  The  Flostile  aircraft  start  in 
either  Iran  or  Iraq,  and  have  varied  flight  pro¬ 
files  including  reconnaissance,  low-altitude 
behavior  and  approach  and  attack  profiles. 


Watchstander  Agents 

The  simulated  watchstanders  were  imple¬ 
mented  as  a  multi-agent  system  where  each 
is  an  agent  (Ferber  1999)  with  a  personality 
and  attributes.  These  agents  interact  with  the 
contacts  and  communicate  with  one  another 
to  coordinate  tasks.  This  allows  users  to 
design  and  run  scenarios  representing  real 
CIC  personnel  and  the  possible  problems 
they  could  encounter. 

Each  action  has  a  probability  of  success 
based  on  the  skill  attribute  of  the  watch¬ 
stander:  basic  (zero  to  six  months  experi¬ 
ence),  experienced  (six  months  to  a  year), 
and  expert  (more  than  a  year).  The  experts 
interviewed  argued  that  a  distinction  should 
be  made  between  skill  and  experience:  Skill 
affected  the  probability  of  success  of  an 
action,  and  experience  affected  both  the  time 
to  complete  it  and  the  degree  of  confidence 
the  agent  had  in  its  results.  The  experience 
levels  were  categorized  as  newly  qualified, 
experienced  (10%  faster),  and  expert  (20% 
faster).  For  watchstanders  that  evaluated 


contacts,  an  evaluation  confidence  attribute 
is  increased  for  each  contact.  The  initial 
value  is  30  and  is  increased  at  a  different 
rate  for  newly  qualified,  experienced,  and 
expert  levels  (by  2,  4,  and  6  respectively),  to 
a  maximum  of  95  for  newly  qualified,  90  for 
experienced,  and  85  for  expert. 

The  fatigue  attribute  controls  the  readiness 
of  the  watchstander.  Based  on  the  interviews, 
three  levels  were  used:  fully  rested  (having 
had  a  minimum  of  five  hours  of  rest  without 
having  performed  heavy  physical  labor  or 
stood  any  watch.),  tired  (a  minimum  of  three 
hours  of  sleep  or  at  most  six  hours  without 
rest  in  a  fairly  demanding  environment),  and 
exhausted  (less  than  three  hours  of  sleep,  or 
having  performed  heavy  physical  labor,  or 
having  performed  duties  over  six  hours  in  a 
demanding  environment.)  Fatigue  decreased 
the  success  probability  for  an  action  and 
increased  the  length  of  time  to  do  it.  Many 
of  the  interviewed  experts  also  argued  for  a 
“decision-maker”  attribute  to  reflea  the  dif¬ 
ferences  among  the  watchstanders  in  how 
long  they  took  to  reach  a  decision.  The  val¬ 
ues  proposed  were  cautious  (for  a  maximum 
of  30  seconds),  balanced  (20  seconds),  and 
aggressive  (10  seconds),  with  a  uniform  dis¬ 
tribution  of  times  up  to  these  maxima. 


Equipment 

The  equipment  used  by  the  watchstanders 
was  modeled  as  separate  software  objeas. 

The  performance  of  seven  key  items  of  equip¬ 
ment  was  simplified  (to  avoid  the  need  to  use 
classified  information)  while  maintaining 
realistic  qualitative  performance.  Equipment 
has  four  readiness  levels  with  associated 
probabilities  of  successful  operation:  fully 
operational  (1.0),  partially  degraded  (0.75), 
highly  degraded  (0.50),  and  non-operational 
(0.0).  In  addition,  if  the  user  has  aaivated  the 
scenario  equipment  failure  option,  any  of  the 
systems  could  randomly  fail  during  the  sce¬ 
nario,  requiring  the  watchstander  agents  to 
troubleshoot  them  until  successful. 
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To  model  the  SPY-t  B  radar  system,  receiver 
operating  characteristics  from  Swerling  11 
statistics  were  used  (Alvarez-Vaquero  1996). 
Data  was  obtained  from  the  AEGIS  SPY-1  B 
Radar  Sphere  Calibration  Test  Procedure  of 
the  Naval  Sea  Systems  Command.  This  gave 
formulas  for  the  carrier-to-noise  ratio  as  a 
function  of  the  size  of  radar  cross-section  of 
a  contact  and  for  the  probability  of  detection 
as  a  function  of  the  carrier-to-noise  ratio. 

The  SLQ-32  radar  detects  electronic  signals 
emitted  by  aircraft  and  shipboard  radar  sys¬ 
tems.  The  simulation  uses  it  to  distinguish 
Iraqi  fighter  and  patrol  aircraft,  Iranian 
fighter  and  patrol  aircraft,  commercial  air¬ 
craft,  friendly  fighters,  friendly  support  air¬ 
craft,  friendly  missiles,  and  hostile  fire-con¬ 
trol  radar  (indicating  either  a  hostile  aircraft 
or  hostile  missile).  The  identification  friend 
or  foe  (IFF)  System  recognizes  friendly  and 
neutral  aircraft  in  five  categories  or  modes. 

A  simplified  IFF  model  was  created  to 
achieve  qualitatively  realistic  performance 

The  Link  11  and  Link  16,  also  known  as  the 
tactical  digital  information  link,  TADIL  A 
and  TADIL  J  respectively,  are  the  main  way 
that  U.S.  and  Allied  military  forces  rapidly 
disseminate  information  about  aircraft  and 
ship  contacts  in  the  operational  area.  These 
data  links  create  a  common  tactical/opera¬ 
tional  picture,  which  enhances  the  military’s 
ability  to  maintain  a  continuous  situational 
awareness  about  the  battlespace.  Their  oper¬ 
ations  in  the  simulation  were  simplified  so 
that  only  contacts  more  than  seventy  nautical 
miles  away  would  be  evaluated.  This  capabil¬ 
ity  modeled  standard  radio  communications. 

In  the  simulation,  surface-to-air  missiles 
were  simulated  with  a  0.70  probability  of 
intercepting  their  target  and  a  range  of 
eighty  nautical  miles.  To  maintain  realism, 
only  two  missiles  can  be  launched  against  a 
target;  if  they  fail,  two  additional  missiles 
can  be  fired.  The  close-in-weapons  system 
(Phalanx)  is  a  twenty-millimeter  shipboard 


self-defense  system  that  contains  its  own 
radar  and  fire  control  system.  In  the  simula¬ 
tion,  it  has  a  range  of  one  nautical  mile  and 
a  0.50  probability  of  hitting  its  target. 

Environment  and 
Doctrine  Attributes 

The  simulation  handles  three  options  for  the 
environment  attribute:  clear  weather,  heavy 
ram,  and  heavy  clutter.  Their  primary  effect 
is  on  detection  and  communications  systems; 
for  instance,  heavy  clutter  reduces  the  proba¬ 
bility  of  detection  by  10%.  The  contact  den¬ 
sity  attribute  controls  the  number  of  con¬ 
tacts  (low,  medium,  or  high).  The  scenario 
threat  level  attribute  (white,  yellow,  or  red) 
affects  classification  of  aircraft  contacts:  The 
higher  the  threat  level,  the  more  likely  the 
team  is  to  classify  aircraft  as  suspect  or  hos¬ 
tile,  and  the  more  common  and  aggressive 
are  the  hostile  contacts. 

An  AEGIS  doctrine  defines  additional  proce¬ 
dures  and  situational  parameters  for  die  CIC. 
The  simulation  implemented  just  the  auto- 
special  doctrine,  a  weapons  doctrine  used  to 
reduce  reaction  time  and  human  errors  when 
a  fast-moving,  anti-ship  cruise  missile  contact 
is  detected  in  very  close  proximity  to  the  ship 
and  poses  an  imminent  danger.  With  Auto- 
Special  Doctrine,  once  a  detected  contact 
meets  the  human-provided  specifications,  the 
ship’s  combat  systems  will  automatically 
engage  the  hostile  missile  with  surface-to-air 
missiles.  Additional  types  of  weapons  and 
identification  doctrine  aid  the  watchstanders 
in  the  performance  of  their  duties. 

Simulation  Logs 

The  simulation  records  every  event  that 
occurs  within  a  scenario  so  that  it  can  be 
later  reviewed  and  analyzed.  Five  record  logs 
are  maintained:  the  scenario  events  log,  the 
watchstander  decision  history  log,  the  CIC 
equipment  status  log,  the  watchstander  per¬ 
formance  log,  and  the  parse/analyzer  log. 
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Gi  Scenario  Events  Log 
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US  detection  on  Tcack  I  50002  UHSOTCESSFUL.  IFF  Bode  l  is  000,  IFF  Bede  2  la  0,  irF  8 
ISCO  analysis  «o  Tcack  I  50011  SUCCESSFUL.  3i<pal  h as  beta  classified  as  CQRERCIA1  AiJ 
R3C  detection  oa  Track  I  5000S  5VCCt53FUl.  Couree  is  13S  DEO  True.  Speed  is  4S1KT3. 

IPS  detection  on  Track  f  30010  SUCCESSFUL.  IFF  Bede  1  Is  000,  ITT  Rode  2  la  0,  ITT  Hod 

TIC  LIB:  analysis  ca  Track  I  SCA20  SUCCESSFUL.  Course  is  203  PEC  True.  Speed  ts  438  R 
P3C  detection  on  Tceck  »  $0013  SUCCESSFUL.  Courae  li  148  DEC  Tree,  speed  is  443KT5. 
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IDS  detection  on  Tcack  I  3000S  SUCCESSFUL.  IIT  node  1  Is  000,  IFF  Bode  2  xs  0,  IFF  Hod 

FTAO  ISCOPKCTLY  dualled  Track  #50011  is  IISXCVH 

ESCO  analysis  oa  Tcack  t  50019  W5UCCES3FUI.  EPRQP.  5Y  EUCO.  Signal  has  been  classifld 

RSC  detection  on  Track  f  S0016  SUCCESSFUL.  Course  xs  156  DEO  Tree.  Speed  Is  439KTS. 

VLS  Hissile  Systea  has  experienced  a  casualty. 

IDS  detection  on  Track  t  50013  SUCCESSFUL.  IFF  Hode  1  Is  000,  Iff  Bode  2  le  0,  IFF  Ko4 
TIC  LUC  analysis  oa  Track  I  500L5  UKVcCZSSFUL.  OR*  5Y  TIC.  Couree  is  136  DZG  Tree 
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FIGURE  3  ,  .  .  ,  .  , 

Example  eueni  log  lor  1  he  scenano  events  log  maintains  a  lngh- 

ihe  simulation  level  record  of  all  events;  an  example  is 

shown  in  Figure  3. 

From  the  logs,  values  are  calculated  for  aver¬ 
age  initial  detection  time  of  aircraft,  average 
initial  classification  time  of  aircraft,  and  aver¬ 
age  correct  classification  time  of  aircraft.  For 
the  individual  watchstanders,  values  are  cal¬ 
culated  for  the  number  of  errors,  number  of 
total  actions  attempted,  and  percentage  of 
errors  in  attempted  actions,  average  action 
durations,  and  average  communications  time. 

FIGURE  4 

Communications 
between  the  uralch- 
slander  agents 


Watchstander  Procedures 

Air  defense  can  be  divided  into  three  phases: 
(1)  contact  detection  and  reporting  (Red 
Crown,  EWCO,  IDS,  RSC,  and  TIC  watch¬ 
standers);  (2)  contact  classification  (FORCE 
TAO,  FORCE  AAWC,  Ship  TAO,  and  Ship 
AAWC);  and  (3)  action  response  (with  the 
FORCE  TAO  and/or  FORCE  AAWC  giving 
orders  to  the  Ship  TAO,  Ship  AAWC,  CSC, 
MSS,  and  IDS).  The  ADC  Simulation  agents 
follow  this  plan  of  action  with  defined  paths 
of  orders.  The  flow  of  information  is  shown 
in  Figure  4  and  the  message  handling  is 
implemented  as  in  Figure  5.  There  are  input- 
message  reception  queues,  message-priority 
processors,  priority  queues,  action  proces¬ 
sors,  and  output-message  transmission 
queues.  Watchstander  agents  place 
order/request  messages  into  another  watch- 
stander’s  input-message  queue  for  process¬ 
ing.  Fifteen  kinds  of  reports  and  six  kinds  of 
orders  are  handled  in  the  implementation. 

Watchstanders  must  also  prioritize  contacts. 
Their  criteria  are  newness  of  the  contact, 
closeness  to  the  ship,  whether  it  is  approach¬ 
ing,  and  duration  since  the  last  examination. 
But  not  all  watchstander  agents  in  the  simu¬ 
lation  rate  these  criteria  the  same  way. 
Furthermore,  watchstanders  must  periodical¬ 
ly  reevaluate  the  same  contacts.  To  model 
the  cognitive  and  decision-making  aspects  of 
contact  classification,  linear  models  were 
used  that  take  a  weighted  sum  of  numeric 
factors:  closing  course,  speed,  altitude,  sig¬ 
nal,  origin,  and  mode.  Different  weights  are 
used  for  each  scenario  threat  level  (white, 
yellow,  and  red).  Four  thresholds  are  used  to 
distinguish  the  conclusions  hostile,  suspect, 
neutral,  unknown,  and  friendly  based  on  the 
advice  of  the  experts  as  well  as  their  actual 
usage  by  operating  naval  forces  (see  Table 
3).  Initial  contacts  are  usually  unknown 
because  only  partial  information  is  available; 
as  more  data  becomes  available,  the  classifi¬ 
cation  changes.  The  most  difficult  and  infre¬ 
quent  classifications  are  hostile  and  friendly. 
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Table  2 


Default  classification  thresholds 

CONTACT 

CLASSIFICATION 

THREAT  LEVEL 

WHITE  THRESHOLDS 

THREAT  LEVEL 

YELLOW  THRESHOLDS 

THREAT  LEVEL 

RED  THRESHOLDS 

Hostile 

>600 

>500 

>450 

Suspect 

500  to  599 

450  to  499 

400  to  449 

Neutral 

400  to  499 

300  to  449 

200  to  399 

Unknown 

-399  to  399 

-399  to  301 

-399  to  199 

Friendly 

<  -400 

<  -400 

<  -400 

Evaluation  of  the  Simulation 

Five  sets  of  questions  were  selected  as  the 
focus  of  testing  the  simulation.  These 
explored  the  influence  of  important  factors 
on  the  performance  of  the  agents  for  the 
RSC,  F.WCO,  and  FORCE  TAO  individually 
and  in  the  CIC  team.  Team  skill,  experience, 
fatigue,  and  the  operational  status  of  the 
SPY-1  B  radar  were  selected  as  the  factors  to 
explore.  For  each  factor  value,  ten  scenario 
runs  were  conducted,  for  170  individual 
tests.  The  experts  suggested  that  the  most 
useful  performance  metrics  were  the  dura¬ 
tion  of  the  actions  of  the  watchstanders  and 
their  error  rate.  Unless  otherwise  indicated, 
tests  assumed  the  watchstanders  were  experi¬ 
enced  in  both  the  skill  and  experience  fac¬ 
tors,  were  fully  rested,  were  balanced  deci¬ 
sion-makers,  and  had  fully  functional  equip¬ 
ment.  It  was  assumed  that  the  external  envi¬ 
ronment  had  medium  contact  density  and 
threat  level  white;  the  hostile  contact  num¬ 
ber  was  low;  and  it  was  clear  weather.  To 
simplify  comparisons  between  tests,  kine¬ 
matic  attributes  of  the  contacts  were  con¬ 
stant,  starting  locations  were  constant,  desti¬ 
nation  points  were  the  same  for  the  same 


,  FIGURES 

starting  point,  no  new  contacts  were  created,  Message  handling  for 
defensive  measures  were  disallowed,  and  IFF  uiatchstander  agents 

was  always  present  for  neutrals.  Tabic  3 
shows  the  average  percentage  change  in  per¬ 
formance  when  varying  the  four  test  para¬ 
meters  from  one  extreme  to  the  other;  the 
error  rate  is  significantly  more  affected  than 
the  other  metrics. 

Another  test  compared  a  scenario  where  the 
FORCE  TAO’s  skill  and  experience  were 
expert  while  the  fatigue  attribute  was 


Table  3 


Average  percentage  change  in  task  time,  communications  time,  and  error  rate  of  three  watch¬ 
standers,  when  varying  key  parameters  over  their  range 


SKILL 

EXPERIENCE 

FATIGUE 

SPY  RADAR  STATUS 

RSC  (Radar  Systems 
Controller) 

3. 27.47 

5, 6, 14 

5.6,14 

7,16, 11 

EWCO  (Electronic 
Warfare  Control  Officer) 

5, 2.35 

>2,9,34 

5.4(4* 

2,10, 15 

FORCE  TAO  (Force 
TacticalAction  Officer) 

2,11,63 

2, 2,23 

4,5, 67 

7,17,15 
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exhausted  (Trial  #1 )  with  a  scenario  in  which 
the  FORCE  TAO’s  skill  and  experience  were 
basic  and  newly  qualified  respectively  while 
the  fatigue  attribute  was  well  rested  (trial 
#2).  For  the  other  watchstanders  in  trial  #1, 
the  skill  and  experience  attributes  were  basic 
and  newly  qualified  respectively,  while  their 
fatigue  attributes  were  fully  rested;  in  trial 
#2,  their  skill  and  experiences  attributes  were 
expert  while  their  fatigue  attributes  were 
exhausted.  Trial  #1  showed  better  perfor¬ 
mance  than  trial  #2  except  in  the  initial  radar 
detection  time,  suggesting  that  the  status  of 
the  team  is  more  important  than  the  status  of 
their  commander. 


Testing  also  included  survey  questions 
administered  to  nine  air-defense  experts  at 
the  ATRC  Detachment  in  San  Diego.  Table  4 
shows  ratings  for  the  simulation  interface, 
where  1  meant  “strongly  disagree”  and  5 
meant  “strongly  agree”.  The  experts  were 
reasonably  satisfied. 

The  experts  were  also  queried  as  to  the  real¬ 
ism  of  the  simulation’s  variation  in  perfor¬ 
mance  with  key  parameters  (Table  5).  For 
instance,  one  question  (the  upper  left  of  the 
table)  asked  whether  it  was  realistic  for  the 
radar  systems  controller  that  performance 
time  improved  and  the  number  of  errors 


Table  4 


Mean  results  for  survey  questions  on  the  interface 


PROPERTY  SURVEY  RESULT  MEAN 

PROPERTY  SURVEY  RESULT  MEAN 

Agent  pop-up  menu 

4.2 

Contact  pop-up  menu 

42 

Simulation  logs  menu 

4.0 

Doctrine  setup  menu 

4.0 

Scenario  external 

attribute  menu 

4.2 

Equipment  setup  menu 

4.0 

Watchstander  attribute 

menu 

4.0 

File  menu 

4-4 

Submenu  items 

Menus  logically  located 

logically  organized 

3.8 

by  functional  area 

3.8 

Menus  easy  to  understand 

3.8 

Pop-up  menus  arranged 
logically 

4-2 

Menus  arranged  logically 

4-0 

Menus  are  intuitive 

3-8 

Methods  for 
tasks  are  reasonable 

3.8 

Tasks  are  understandable 

4.0 

Table  5 


Mean  results  for  survey  questions  on  the  simulation  realism 

ISSUE  AS  TO  REALISM  SURVEY  RESULT  MEAN  ISSUE  AS  TO  REALISM  SURVEY  RESULT  MEAN 

RSC  skill  change 

6.n 

RSC  experience  change 

6.00 

RSC  fatigue  change 

5-33 

RSC  SPY  radar  change 

5.00 

Team  interaction  with 

RSC  performance 

5.00 

EWCO  skill  change 

6.22 

EWCO  experience  change 

5.89 

EWCO  fatigue  change 

544 

EWCO  SLO-32  change 

544 

Team  interaction  with 

5-44 

EWCO  performance 

FORCE  TAO  skill  change 

6.00 

FORCE  TAO  experience  change 

5-78 

FORCE  TAO  fatigue  change 

5.56 

FORCE  TAO  decision¬ 

maker  change 

4-33 

Team  interaction  with 

Team  interaction  with 

FORCE  TAO  performance 

5.22 

FORCE  TAO  decision¬ 

maker  type 

444 

Realism  of  Trial  *1 

5.00 

Realism  of  Trial  #2 

4.67 
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decreased  when  the  Experience  level 
increased.  For  these  answers  the  scale  was  1 
(strong  disagree)  to  7  (strongly  agree).  These 
results  were  also  encouraging. 

Conclusions 

The  Air  Defense  Commander  (ADC) 
Simulation  successfully  simulates  the  mental 
processes,  decision-making,  cognitive  attrib¬ 
utes,  and  communications  of  an  eleven- 
member  CIC  air  defense  team  performing 
their  duties  including  under  stressful  condi¬ 
tions.  The  ADC  Simulation  should  assist  air- 
defense  trainers  in  gaining  insight  into  the 
degree  to  which  watchstander  skill,  experi¬ 
ence,  fatigue,  type  of  decision-maker,  and 
environmental  attributes  influence  the  per¬ 
formance  of  the  individual  as  well  as  the 
CIC  watch  team.  The  simulation  offers 
enough  flexibility  and  options  for  a  user  to 
study  the  effect  of  a  variety  of  factors  on 
performance.  Future  research  directions 
include:  a  networked  simulation;  more 
detailed  human  models;  aircraft  contacts  as 
agents;  more  detailed  log  parsing  using 
XML;  implementation  of  additional  doc¬ 
trines;  alternate  scenario  locations;  more 
detailed  treatment  of  radar  systems;  a  more 
detailed  study  of  metrics  for  watchstander 
performance;  a  capability  to  replay  previous 
scenarios  and  portions  of  them;  and  a  capa¬ 
bility  to  build  scenarios.  The  simulation  also 
can  be  adapted  for  wargaming,  and  provides 
a  useful  set  of  tools  for  building  other  simu¬ 
lations  of  human  teamwork. 
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